- Generalizing overall treatment effects is often problematic
- Subgroup analyses rarely adequately powered
Subgroup analyses can be divided into 4 categories (
Risk modeling
For all patients we observe covariates \(x_1,\dots,x_8\), of which \(4\) are continuous and \(4\) are binary. More specifically,
\[x_1,\dots,x_4 \sim N(0, 1)\] \[x_5,\dots,x_8 \sim B(1, 0.2)\]
Generate the binary outcomes \(y\) for the untreated patients (\(t_x=0\)), based on
\[P(y\:\vert\:x, t_x=0) = g(\beta_0 + \beta_1x_1+\dots+\beta_8x_8) = g(lp_0),\]
where \[g(x) = \frac{e^x}{1+e^x}\]
For treated patients, outcomes are generated from:
\[P(y\:\vert\:x, t_x=1) = g(lp_1)\]
where \[lp_1 = \gamma_2(lp_0-c)^2+\gamma_1(lp_0-c)+\gamma_0\]
Figure 1: Linear and quadratic deviations from the base-case scenario of constant relative effect (OR=0.8)
Figure 2: Linear and quadratic deviations from the base-case scenario of constant relative effect (OR=0.8)
Base-case scenario
Deviations
Finally, we consider 3 additional scenarios of interaction of individual covariates with treatment. These scenarios include:
Constant treatment effect \[E\{y\:\vert\:x,t_x\} = P(y\:\vert\:x, t_x) = g(\beta_0+\beta_1x_1+\dots+\beta_8x_8+\gamma t_x)\] Absolute benefit is estimated from: \[\hat{f}_{\text{benefit}}(lp\:\vert\:x, \hat{\beta}) = g(lp) - g(lp+\hat{\gamma}) \]
Linear interaction \[E\{y\:\vert\:x, t_x, \hat{\beta}\} = g\big(lp+(\delta_0+\delta_1lp)t_x\big)\]
Predict absolute benfit from: \[\hat{f}_{\text{benefit}}(lp\:\vert\:x, \hat{\beta}) = g(lp) - g\big(\delta_0+(1+\delta_1)lp\big)\]
Non-linear interaction \[f_{\text{benefit}}(lp\:\vert\:x,\hat{\beta}) = \hat{f}_{\text{smooth}}(lp\:\vert\:x, \hat{\beta}, t_x=0) - \hat{f}_{\text{smooth}}(lp\:\vert\:x, \hat{\beta}, t_x=1)\] We use restricted cubic spline smoothing wit 3, 4 and 5 knots.
Root mean squared error
Assuming that \(\tau(x)=E\{y\:\vert\:x, t_x=0\} - E\{y\:\vert\:x,t_x=1\}\) is the true benefit for each patient and \(\hat{\tau}(x)\) is the estimated benefit from a method under study, the ideal loss function to use for the considered methods would be the unobservable root mean squared error \(E\big\{(\hat{\tau} - \tau)^2\:\vert\:x\big\}\).
We will estimate the RMSE from \[\text{RMSE}=\frac{1}{n}\sum_{i=1}^n\big(\tau(x_i) - \hat{\tau}(x_i)\big)^2\]
C-for-benefit
Calibration for benefit
Figure 3: RMSE of the considered methods across 500 replications calculated in a simulated superpopulation of size 500,000.
Figure 4: RMSE of the considered methods across 500 replications calculated in a simulated sample of size 500,000. Sample size 17,000 rather than 4250 in Figure 3.
Figure 5: RMSE of the considered methods across 500 replications calculated in a simulated sample of size 500,000. AUC is 0.85 instead of 0.75 in Figure 3.
Figure 6: Discrimination for benefit of the considered methods across 500 replications of the base case scenario calculated in a simulated superpopulation of size 500,000.
Figure 7: Calibration for benefit of the considered methods across 500 replications of the base case scenario calculated in a simulated superpopulation of size 500,000.
Data from GUSTO-I trial
Patients with acute myocardial infarction were randomized to:
Outcome: 30-day mortality
In line with previous analyses (
Figure 8: Calibration for benefit of the considered methods across 500 replications of the base case scenario calculated in a simulated superpopulation of size 500,000.
Appendix A: Integrated calibration indexHarrell: \[\text{E}_\text{max}(a, b) = \text{max}_{a \leq \hat{P} \leq b} |\hat{P} - \hat{P}_c |\]
Austin et al: \[\text{ICI} = \int_0^1f(x)\phi(x)dx\]